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1 INTRODUCTION 



, Decentralized control has been a challenging area of open research for over three decades. However, these problems 
I— i' are still intractable in general. In one of the earliest works, it was shown that for a linear system subject to quadratic 
' cost and Gaussian noise, complex nonlinear controllers could outperform any given linear controller (see [1]). 

Centralized controller design problems assume a classical information pattern where each system has access to the 
i-S^ , information of all other systems. However, in decentralized problems, different systems have access to different 
' information sets and their decisions are based on local measurements. Solving the problem in the presence of this 
^ " constraint is a much harder problem. 

' One approach has been to classify specific information patterns leading to linear optimal controllers. In [2], sufficient 
T-H [ conditions are given under which optimal controllers are linear for the LQG problem. The one-step delay LQG 
^ ■ problem was solved in [3] . An important result was given in [4] which showed that for a new information structure that 
' was called partially nested, the optimal controller is linear in the information set. In [5], stochastic linear quadratic 
, control problem was solved under the condition that all the subsystems have access to the global information from 
some time in the past. In [6], it was shown that the constrained linear optimal decision problem for infinite horizon 
linear quadratic control, can be posed as an infinite dimensional convex optimization problem, given that the system 
Tj" , considered is stable. A solution to the state feedback problem, using a duality result for distributed estimation and 
' control under partially nested information pattern, was given in [7] . 

CN 

' , I I In this paper, we consider a decentralized system over an acyclic graph. A quadratic cost is to be minimized under 
^ ' the partially nested information structure. The goal is to explore the structure of the distributed controllers. 

This problem is known to have a linear optimal policy, see [4] and [8]. However, most existing approaches do not 
^ ^ provide explicit optimal controller formulae and, in addition, the order of the controller can be large. A solution to 
- - - this problem has been first given in [9] , where the resulting controller is of high order which makes the implementation 
difficult. Moreover, some work has been focused on finding numerical algorithms to these problems [10], [11]. 



State-space solutions to the two-systems state feedback 'H2 version of this problem was given in [12]. Another instance 
of the problem with a partial output feedback structure has been considered in [13], in which one system measures 
his state directly and the other system uses a noisy measurement of his own state. However, no previous work has 
provided a solution to the complete output feedback problem for N interconnected systems, not even for the case 
N = 2. hi this paper, we propose a methodology to attack the problem for the case of two interconnected systems 
to understand the structure of the controller. 



Notation 



We denote a matrix partitioned into blocks by ^ = [^ij]: where Aij denotes the block matrix of A in block position 
(«,j). A discrete-time stochastic process x{Q),x{l),x{2), ... is denoted by {x(t)}. For vectors y{Q),y{l), ...,y{t), we 

r 1 ^ 

let 2/(0 : t) — j/^(0) y'^(l) ■ ■ ■ y'^{t) ■ 5{t) denotes the Dirac delta function. 
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2 Problem Statement 



Consider a linear system composed of N interconnected subsystems 

x{t + 1) = Axit) + Bu{t) + w{t), 
y{t) = Cx{t)+v{i), 

where the system matrices are partitioned into blocks as A = [Aij], B = [-Bij], C = [Cij], i,j = 1, N , and x, u, 
and y are the overall state, input, and output vectors. These vectos are partitioned as 



x{t) = 



xl{t) xj{t) ••• xlit) 



u{t) = uJit) u'^it) ■ ■ ■ Ujf{t) 



yit) 



where Xi{t) is the state of subsystem i, yi{t) is the noisy measurement made by subsystem i and Ui{t) is the control 
input to the subsystem i at time t. The interconnection structure can be described as follows. If the state of system 
j at time step t (i.e., Xj{t)) affects the state of system i at time step t + 1 (i.e., Xi{t + 1)), then Aij ^ 0, otherwise 
Aij = 0. This structure can be represented by a directed graph of order N. The graph has an arrow from node j to 
i if and only if Aij ^ 0. 

Here, we assume system dynamics over an acyclic graph. For a directed acyclic graph it is natural to order the nodes 
so that the adjacency matrix becomes lower-triangular. Thus, we shall consider a lower-triangular A matrix for the 
remainder of this paper. Furthermore, we assume that the matrices B and C have the same sparsity structure: 
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The interconnection structure for a two-systems chain, and a three-systems graph has been shown in Fig. 1 and Fig. 
2 respectively. Assume {w^t)} and {v{t)} are sequences of Gaussian vectors with zero mean values and covariances 



E 


w{k) 




wil) 




y{k)_ 




v{l)_ 



= S{k - 1) 



W U 
V 



Furthermore, the covariance matrices are partitioned into appropriately sized blocks 

W = [Wij], V = [Vij], for i,j = l,...,N. 



(4) 
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Fig. 1. Two-Systems Chain 




Fig. 2. Three-Systems Graph 
Vj will denote the jth block column of V. We want to minimize the infinite-horizon quadratic cost 



M-l 



lim ^ V E 



t=0 



x{t) 


T 


Q S 




x{t) 


uit) 




R 




uit) 



(5) 



where 



g s 

R 



>~ 0. The weighting matrices are also partitioned as Q = [Qij\,R = [Rij],S = [S'y], i-j = l,...,iV 



Sj will denote the jth block column of S. In this system, the dynamics of subsystem j propagates to subsystem i if 
j < i. Clearly, if all systems have access to the global measurements, the information structure would be classical, 
and the optimal linear controller could be obtained from the LQG control theory. However, we consider the case in 
which each system i has access to its own measurements up to time t — 1, i.e. yi{0 : t — 1). Furthermore, system i has 
access to the information set that system j has access to if the decision that system j makes affects the information 
set of system i. This assumption implies that the information pattern is partially nested. This information pattern 
is not classical and is one of a few non-classical information patterns for which the optimal policy is known to be 
unique and linear in the associated information set [4] . 

Hence, we are interested in finding the controllers restricted to the following structure: 



wi(i)=Mi(yi(0:t-l)), 

U2{t) = M2(j/i(0 : i - l),j/2(0 : t - 1)), 

ujv(i) = Mjv(yi(0 : t - 1),. . .,j/jv(0 : t - 1)), 



(6) 



where /Ui, . . . , /xjv denote linear functions in all their variables. The objective is to find the stabilizing controllers that 
minimize the cost criterion (5) while restricted to the information constraints in (6). 

3 Solution for Two Interconnected Systems 

For the two-systems problem, (Fig. 1), control inputs are constrained on the form: 

ui{t)= fii{yi{0:t-l)), 

U2{t)= H2{yi{0:t-l),y2{0:t-l)). 

Equivalently, controllers can be described as the following linear dynamical systems: 

rii{t + 1) = FiT^iit) + Giyiit) 
ui{t) = -Hirjiit) 



m{t + 1) = F2r]2{t) + G2 
U2{t) = -H2m{t) 



.2/2 W. 
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To simplify notation, we define the block rows of C as 







Cii 


C2 




C21 C22 



and the block columns of B as 



BAB2 








-B21 


B22 _ 



We shall now state the optimal controller synthesis for the two-system problem as the main result of this section. 



3.1 Main Result 



Theorem 1 Consider the optimal control problem 



M-l 



min lim — E 

Ml,U2 M->-00 M ' 

t=0 



xit) 


T 


Q S 




x{t) 


u{t) 




R 




u{t) 



subject to 
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_X2{t). 
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«i(t) = Mi(2/i(0:t-l)) 

«2(i) = M2(yi(0:t-l),t/2(0:i-l)) 



Suppose there exist stabilizing solutions P, 11, Pi, 112, to the Riccati equations 



P ={A - KC)P{A - KCf 

+W -KU - U'^K'^ + KVK'^ 
n ={A - BLY'li{A - BL) 

+Q - L'^S^ -^SL + L^RL 
Pi ={A - KiCi - B2L2)Pi{A - KiCi - B2L2f 

+W - KiU'^ - UKl + KiVKl 
JI2 ={A - KiCi - B2L2fU2{A - KiCi - B2L2) 
+Q — L2S2 — S2L2 + L2R22L2 



(7) 
(8) 
(9) 
(10) 



4 



where 



K = {APC^ + U){CPC^ + V)-\ 
L = {B'^UB + R)-'^{A^IIB + Sf, 
Kr = {{A - B2L2)PiCj +U){CiPiCj + V)-\ 
L2 = (BjnaSs + i?22)-'((^ - KiCifn2B2 + S2f, 
S12 



(11) 
(12) 

(13) 
(14) 



52 = 



'22 



'w u 




IF V 





KiCPC^ + V)K'^ KCPCJ + Vx 
CiPC^K^ + CiPCl + Vii 



(15) 



Then, the optimal controllers are given by: 



Ui{t) 




itiit) 


+ 
















m 



(16) 



zit + 1) = {A- KiCi)z{t) + Bu{t) + Kiyi{t) 



-Lz{t) 



z{t +1) = {A- KC)z{t) + Bu{t) + Ky{t) 



zit) - z{t) 

-L2Z{t). 



3.2 Optimal Controller Derivation 

3.2.1 State and Controller Estimation 

In this part, we will present the optimal estimators involved in the controller design. Note that since the controllers 
ui and U2 don't have the same information, their estimators of the system will be different. 



^(t)=E{x(t)|2/(0:t-l)}, 
x{t) =x{t) -x{t). 

x{t) is given by the Kalman filter ([14]) 

x{t + 1) = Ax{t) + Bu{t) + K{Cx{t) + v{t)) 

where K is obtained from (7) and (11). Introduce 

uj{t) = K{Cx{t) + v{t)), 
u{t) = Cix{t)+vi{t), 
z{t) = x{t). 

By construction, z/(t) is orthogonal to z{t). Now we have that 
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Also, let 



K{CPC^ + V)K'^ KCPCl + Vi 



z{t + 1) = Az{t) + Bu{t) + u){t), 
yi{t) = Ciz{t) + u{t). 



u{t)=-E{u{t)\yi{0:t-l)}, 
u{t) =u{t) -u{t), 

z{t) = n4t)\yi{0-t-i)}, 

z{t) = z{t) - z{t) - z{t), 

where z{t) is given by a Kalman filter 

z{t + 1) = Az{t) + Bu{t) + Ki{Ciz{t) + u{t)). 
and the estimation error dynamics are given by 

z{t + 1) = {A- KiCi)z{t) + Bu{t) + ^{t) 

with jl){t) = —Kii'{t) +uj{t), and 



E{^(i)V^(t)} 



W U 



I -K, 



(17) 



Note that z{t) is uncorrelated with co{t),i'{t), since z{t) and z{t) are uncorrelated with w{t),i'{t). The gain Ki 
depends on the dynamics of z, which in turn depends on u in (17), and can be found once we have the dynamics of 
u. 



3.2.2 Cost Decomposition 
Note that 
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The components in the second equality above are uncorrelated, so 
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and hence, we decompose the cost function into three components as J = Jz + Jz + Jx, with 



^ M-1 

jz= lim — V E 

t=0 



M-1 



lim ^ y E 



t=0 



m 


T 


Q 


s 




u{t) 






R 




m 


T 


Q 


s 




iiit) 






R 





m 

u{t) 

m 

u{t) 



and 



M-1 



J^= lim ^ y E x^{t)Qx{t). 

M-)-oo M ^ 



t=0 



Jx is independent of the controller, and hence, nothing can be done about it. Minimizing Jz and Jz is done with 
respect to independent decision variables, u{t) and u{t), repectively. 

3.3 Finding Controller ii 

Seeking the optimal policy that minimizes the cost Jz can be cast as the following optimization problem 
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min lim 

u M^oc M 



- EE 
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subject to z{t + 1) = Az{t) + Bu{t) + Ki{Ciz{t) + v{t)). 



(18) 



u{t) is a function of i/i(0 : f — 1) by construction. Hence, it can be a function of z{t). This is clearly a centralized state 
feedback LQ problem. Under stabilizability conditions, the optimal controller is given by u*{t) = —Lz(t), where L 
is obtained from the Riccati equation (8) ([14]). 

3.4 Finding controller u 

Note first that Ui{t) depends only on yi(0 : f — 1), so Ui{t) = Ui{t). Thus, Ui{t) = and 



u{t) 







Now the optimal controller synthesis for the cost Jz becomes: 
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U2 M-s-oo M 
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subject to z{t +l) = {A- KiCi)z{t) + B2U2{t) + ip{t) 

Clearly, this problem is also a centralized state feedback control problem since U2 measures z. Therefore, under 
stabilizability conditions, the optimal controller is given by 



U*2{t) = -L2Z{t), 

where L2 = i?~^Bjn2, and 112 is given by the Riccati equation (8) ([14]). 



(19) 
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3.5 Finding the Estimator Gain K\ 

First, take the optimal controller (19) and plug it in the expression of the Kalman filter error equation (17): 

~z{t + 1) = (^ - KiCi - B2L2)z{t) + ^P{t). (20) 

Thus, the error dynamics of z give rise to the Riccati equation (9) ([14]), and Ki is the solution obtained from (9) 
and (13). 



4 Discussion 

We can see that the controller is decomposed into two parts, 



'ui{tj 




'ui{t) 


+ 
















where 

_Mt). 

is a state feedback law based on z{t). z{t) is an estimate of the centralized estimate of the state x{t), based on the 
output history j/i(0 : t — 1) which is available to both controllers. The additional information controller 2 has access 
to, 2/2(0 it—l), is used to to minimize the part of the cost (J^) that can not be observed with respect to yi(0 : t — 1). 
This is given by U2{t). 

The dependence of the filter (17) on part of the controller, w,2, is essentially what makes the problem different from 
centralized control, where the estimator becomes independent of the controller. This is also why we get a coupling 
between the estimator gain Ki and the controller gain L2 in the Riccati equations (9)-(10). 

We note that the Riccati equations (7)-(8) can be solved independently. The Riccati equations (9)-(10) are coupled 
and depend on the output from (7)-(8). The coupling looks complicated, but using the fact that the equalities in 
Riccati equations can be written as inequalities ([15]) , and by Schur complementing, we can obtain a set of linear 
matrix inequalities that can be efficiently solved. 

Also, in principle, the solution can be easily extended to the case of N interconnected systems by induction. We 
simply take the A''th system as the "second system" considerd in this paper, and the rest of systems 1, A'' — 1 
to be the "first" system. Then we proceed iteratively by taking system — 1 as the "second" system and systems 
1, A^ — 2 as the first, and so on. However, the coupled Riccati equations might give rise to a complicated numerical 
solution. 
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